Camera system comprising a camera, camera, method of operating a camera and method for deconvoluting a recorded image

ABSTRACT

A system and camera wherein the camera comprises in the light path a diffuser ( 4 ). The system or camera comprises a means ( 6 ) to modulate the diffusing properties of the diffuser ( 4 ) on an image projected by the lens on the sensor during exposure of the image. To the captured blurred image ( 10 ) an inverse point spread function is applied to deconvolute ( 24 ) the blurred image to a sharper image. Motion invariant image can so be achieved.

FIELD OF THE INVENTION

The invention relates to a camera system comprising a camera with a lensand an image sensor.

The invention also relates to a camera with a lens and an image sensor.

The invention also relates to a method of operating a camera comprisinga lens and an image sensor.

The invention also relates to a method for deconvoluting image datarecorded by a camera comprising a lens and an image sensor.

BACKGROUND OF THE INVENTION

Conventional image sensors, such as CMOS and CCD sensors, integrate alllight that impinges on them during the exposure time. This providessharp images of static objects, but results in spatial blur for objectsthat move while the shutter is open. Objects that are not in focus, i.e.not positioned in the focal plane are also blurred. The so-called motionblur is proportional to the exposure time and object velocity. Motionblur is particularly troublesome when a camera operates under low lightlevel conditions. Under such circumstances, long exposure times aredesired to attain sufficiently high signal-to-noise levels such that thedark areas of a scene can be imaged adequately. Consequently, manycameras suffer from a classic trade-off between motion blur and dynamicrange. The exposure times need to be long to capture enough light, butneed to be small so as to reduce motion blur. Within the framework ofthe invention the word camera comprises cameras for taking photographsas well as cameras for video purposes.

A camera and method to reduce blur of objects that are not positioned inthe focal plane are known from an article by Nagahara et al “FlexibleDepth of Field Photography”, H. Nagahara, S. Kuthirummal, C. Zhou, andS. K. Nayar, European Conference on Computer Vision (ECCV), October,2008.

In Nagahara et al a camera for taking photographs is shown in which thedistance between the sensor and a fixed focus lens is varied. The sensoris swept over a distance during the exposure time. The sweeping distanceis arranged to sweep a range of scene depth ranges in order to increasethe depth of field. The prior art camera disclosed in Nagahara et alreduces out-of-focus blur. To reduce the out-of-focus blur the sensor isswept along the optical axis to cover certain depth ranges. This conceptis also called the ‘sweeping focus camera’.

The sweeping of the focus provides for a compound image, in effect beinga combination of a number of images at various focal depths. A pointspread function (PSF) characterizing the blur caused by the sweepthrough various focal positions can be calculated. A point spreadfunction is, in effect, the image a point of an object would make on thesensor. For an object completely in focus the point spread would bezero, and thus the PSF would be a Dirac function. The Fourier transformof this function would be a constant for all frequencies. For a pointnot in focus the PSF is a spread-out function, for an object in motionwhile the camera is fixed, the PSF would be spread out over a distancedue to the motion. From the PSF one can calculate an inverse pointspread function (IPSF). Deconvoluting the compound image with theinverse point spread function allows a sharp image to be obtained and anincreased depth of field is obtained. This is due to the fact that, whenthe sensor is swept, the PSF for static objects at various distancesbecomes to a considerable degree the same. Thus, deconvolution theoriginal image with one and the same IPSF would allow a sharp image atall distances, or at least an increased range of distance and the anincreased depth of field is obtained for static objects.

Although out-of-focus blur and the reduction thereof may be and isimportant, a major problem, as explained above, exists and remains formoving objects, namely the motion blur, especially for larger exposuretimes.

Motion blur can be inverted by means of video processing. This isachieved by motion estimation and inverse filtering along the motiontrajectory. This is known for instance from U.S. Pat. No. 6,930,676. Inpractice, however, the results of such a procedure suffer frominaccurate motion vectors, particularly for occlusion areas. One has toknow the motion trajectory and deduce motion vectors from them to beable to do the inverse filtering. In many stand-alone cameras used inprofessional applications, motion vectors may not be available at all.For example, the recordings of many cameras used for surveillance oractivity monitoring merely provide input to computer-vision-basedanalysis procedures (e.g., automatic detection of suspicious objects,fall-detection for elderly, etc). In these scenarios, the quality of theraw input frames is a determining factor for the performance of thedetection system. Sufficiently accurate motion vectors may not beavailable on-the-fly within the camera and post-processing of recordedvideo is not an option in real-time monitoring systems. For a camerathat takes a single snapshot it is fundamentally impossible toaccurately determine motion vectors. At occlusion areas estimation ofmotion is also extremely difficult and inaccurate, if at all possible.At low light conditions the problems increase, due to the lack of light.Another method of getting rid of motion blur is to have the camerafollow the moving object. However, this also has a number of significantdisadvantages. Although the moving object is not blurred, everythingelse is. Furthermore, one has to know the direction as well as the speedof the object to accomplish such facts. This methods is thus onlypossible in situation where a fairly accurate determination of the speedand direction of movement can be made, for instance with formula 1 racesor a ski jump, where the direction of motion is fairly accurately knownin advance as well as an relatively accurate esiamate of the speed maybe made in advance.

Second, most traditional cameras feature an adjustable shutter andaperture that windows the light coming through the lens in the temporaland spatial dimensions. These can typically be characterized as boxfilters (i.e. a constant sensitivity over a finite interval),corresponding to a sinc modulation in the corresponding temporal andspatial frequency domains. As a result, some high frequencies are fullysuppressed during acquisition and cannot be recovered during inverse FIRfiltering even when perfect motion information would be available. Inpractice, inverse filtering should be done with utmost care to preventthe amplification of noise and the introduction of artefacts.

In International Patent Application WO 2010/131142 a system is describedin which a sweeping focus set-up is used to accomplish motion invariantimaging. This is achieved by sweeping the focus fast by moving thesensor or lens, of changing the focus of the lens.

Since a sweeping focus is used the acquired image is sharp throughoutthe focal sweep range, background and foreground are sharp. To the humaneye such an all-in-focus image often looks unnatural.

In short, various known ways for reducing motion blur in an image havetheir shortcomings.

SUMMARY OF THE INVENTION

It is an object to the invention to reduce motion blur in an alternativeway.

To this end the system and camera according to the invention ischaracterized in that the camera comprises in the light path a diffuser,the system or camera comprising a means to modulate the diffusingproperties of the diffuser on an image projected by the lens on thesensor during exposure of the image.

The method of operating a camera according to the invention ischaractreized in that the camera comprises in the light path a diffuser,and during the image acquisition the diffusing properties of thediffuser are modulated.

A method for deconvoluting image data according to the invention ischaracterized in that the camera comprises in the light path a diffuser,and during the image acquisition the diffusing properties of thediffuser are modulated and an inverse point spread function is appliedto the aquired image to deconvolute the acquired image.

Motion blur is caused by movement of object in a direction perpendicularto the optical axis, for instance in a horizontal or vertical direction.This motion provides for an apparent motion of the object on the sensorduring the exposure, which smears out the image of the moving object inthe recorded image which leads to motion blur. Modulating the diffuserseems only to be contraproductive by introducing additional blurring.However, the blurring of the image due to the modulation of the diffusercan be undone by deconvolution of the image by using the appropriateinverse point spread function (IPSF) for the blur kernel due to thedynamic diffuser, equivalent to the inverse of the point spread functionfor the blurring caused by the dynamic diffuser.

The inventors have realized that introduction of blurring of the imageby a modulated diffuser placed in the light path can be in fact be usedto effectively counteract motion blur. The blur kernel introduced by thedynamic diffuser becomes to a practical degree the same for a range ofobject velocities. The PSF is therefor to a practical degree the samefor a range of object velocities. This allows for a motion invariantimaging by using an IPSF which provides a sharp image for a range ofobject velocities by introducing dynamic blurring of the image. Thecamera may have a fixed focus during the image acquiring.

It is remarked that the dynamic diffuser can be placed anywhere withinthe light path, in front of the lens, or between the lens and the sensorof the sensor. Placing the dynamic diffuser in front of the lens allowsexisting cameras to be converted into cameras according to theinvention. Placing the dynamic diffuser in between the camera and thesensor has an an advantage that the position of the dynamic diffuser canbe accurately determined.

Dynamic diffusing can be achieved for instance by using an electricallymodulated diffuser and a diffuser driver which is responsible for thechanging of the diffusive properties according to a controlling signalfrom a system controller, wherein the system controller synchronizes thedynamic modulation of the diffuser with the camera shutter.

Alternatively a diffuser with static properties can be used and theposition of the diffuser can be changed during the exposure time, bymoving the diffuser from a position close to the sensor to a more remoteposition while the shutter is open. In this embodiment the propertiesfor the diffuser as such are not modulated. However, by modulating theposition of the diffuser during the exposure time the diffusing effectof the diffuser on the image is modulated and thereby the effect of thediffuser becomes dynamic.

Preferably, however, the modulated diffuser has a fixed position and thediffusing properties of the diffuser are modulated. Movement of parts ofthe camera may cause vibrations which may cause blurring that cannot becounteracted by deconvolution an and may also cause, in due time,friction or relaxation, all potentially having negative effects.

The invention also relates to a system for recording images comprising acamera, further comprising a deconvolutor for deconvolution of arecorded image, wherein the camera comprises in the light path a dynamicdiffuser, the system or camera furthermore comprising a means tomodulate the properties of the diffuser from transparent to diffusingduring the image integration and wherein the recorded image isdeconvoluted with an inverse point spread function.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and advantageous aspects will become apparentfrom exemplary embodiments that will be described using the followingFigs.

FIG. 1 illustrates the principal components of a camera system of theinvention;

FIG. 2 illustrates an algorithm, system and method to filter a recordedimage.

FIG. 3 illustrates various steps in the method of the invention.

FIG. 4 illustrates the effect of a dynamical diffuser.

FIGS. 5A and 5B illustrates graphically the relation between thediffusion effect and the the voltage applied to the diffuser.

FIGS. 6, 7 and 8 illustrate respectively a space-time representation ofthe sensor integration area, the effective system blurring kernel, andthe system blurring kernel on the logarithmic plot.

FIGS. 9 to 11 illustrate the system blurring kernel for a moving object

FIGS. 12 and 13 illustrate the system blurring kernel for a fast movingobject, wherein the object speed exceeds the blur speed.

FIGS. 14A and 14B illustrate two embodiments of a dynamic diffuser

FIGS. 15A and 15B illustrate two embodiments of electrodes for a dynamicdiffuser.

The figures are not drawn to scale. Generally, identical components aredenoted by the same reference numerals in the figures.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The invention is based on the insight that by changing the cameraparameters during the exposure time, the characteristics of the recordedimage can be modified. This is exploited to design a camera of which themotion blur is, within a practical speed range, almost independent ofthe motion of the objects and preferably one of which the frequencybehaviour is such that the recorded signal is better suited for inversefiltering. This allows sharp images to be generated from longer exposuretime recordings without the need for motion estimation. In other words,the camera in accordance with the invention can provide sharp imageswith high SNR even for very challenging optical imaging conditions:objects moving at various, unknown velocities under low illuminationlevels. To this end a dynamic diffuser is present in the light path andthe diffusing properties of the diffuser are modified from transparentto diffusing during the exposure time preferably in synchronicity withthe exposure. This can be done by placing a diffuser at a fixed positionwithin the light path and change dynamically during the exposure thediffusing properties of the diffuser, or, by using a diffuser with fixeddiffussing properties and moving the diffuser during the exposure. Ofcourse any combination of the two, to increase the range of change indiffusion, may also be possible.

FIG. 1 illustrates various embodiments of the invention.

Object P is, through a lens 2, focussed. The focussed image is in FIG. 1illustrated by the point where the light rays coming from the objectcross. In FIG. 1 the camera has a dynamic diffuser 4 in the light path,which in this embodiment is situated between the lens 2 and the sensor3. The difffusing action of diffuser 4 is, during the exposure time,dynamically adjusted. The concept of a traditional camera is thusmodified in that its optical path a modulated diffuser 4 is positionedwhich is changing its diffusive properties (from transparent todiffusive) during the image integration. This concept is illustrated inFIG. 1. The system comprises the elements of a traditional camerasystem, and in addition a modulated diffuser, and in this example adiffuser driver 6 which is responsible for the changing of the diffusiveproperties according to a controlling signal, provided by a systemcontroller 5 which synchronizes the action of the diffuser with thecamera shutter.

The bundles of rays that exit the diffuser 4 schematically illustrate adiffuser in action, i.e. a diffuse bundel of light exiting the diffuser.In this figure a set-up is provided in which the dynamic diffusingproperties of a diffuser, situated at a fixed position, vary.Alternatively one can use a diffuser with fixed properties and move thediffuser from a position near the sensor (at which position the diffuserhas little effect) to and fro the sensor during exposure. The diffuserwill lead to an image that is blurred due to the diffusing effect of thediffuser. Instead of a sharp point a blurred point is imaged on thesensor 3. The shape of the blurred point is also called the blur kernel.In FIG. 1 the object is assumed to be stationary.

The inventors have realized that introduction of blurring of the imageby a modulated diffuser placed in the light path can be in fact be usedto effectively counteract motion blur. The blur kernel introduced by thedynamic diffuser becomes to a high degree the same for a range of objectvelocities. The PSF of a point that is imaged on the sensor in a blurdue to the diffusing action of the diffuser is to a high degree the samefor a range of object velocities. This allows for a motion invariantimaging by using an IPSF which provides a sharp image for a range ofobject velocities by introducing dynamic blurring of the image.

FIG. 2 illustrates how a recorded blurred image is deconvoluted toprovide a sharp image. FIG. 2 illustrates a method for deconvoluting arecorded image and a system for recording image data and deconvolutingof recorded image data. The image is recorded by the camera, comprisinga lens 2 and a sensor 3. In between the lens 2 and the sensor a diffuser4 is positioned. A diffuser driver 6 regulates the diffusing function ofthe diffuser 4 during an exposure. The image data of the sensor 3 areread by a reader. The corresponding image 10 is schematically shown as ablurred image. The recorded blurred image 10 is deconvoluted indeconvolutor 20 to produce a sharpened image 11.

The system comprises a deconvolutor 20 to deconvolute the image data ofblurred image 10. The method deconvolutes the recorded image data ofimage 11 by performing a deconvolution operation on the recorded image.For ease of understanding the algorithm for and method of deconvolutionis shown in a number of steps. The first step 22 is to calculate orestablish a point spread function PSF. In simple embodiments the PSFfunction for blurring due to the action of the diffuser is calculatedfor a static object independent of any other parameter. Since, asexplained below, for a large range of speeds the PSF functions arenearly independent of the speed for embodiments of the invention, anIPSF for a static object will be a good first order approximation for anoptimal PSF for a wide range of speeds. In more advanced embodiments thedistance of the object or the speed of the object may used to fine-tunethe PSF and thereby the IPSF. The distance of an object can for instancebe recorded by the camera. Most cameras have some sort of autofocus thatallows a determination of an object distance. As explained above it hasbeen found that even if an object is not in focus in the middle of thesweep motion invariant imaging is very well possible. However, althoughusing a single average PSF function for a static object will give goodresults, somewhat improved results may be obtained by fine-tuning thePSF by making the PSF dependent on the object distance and possiblyfurther parameters, such as shutter time. This will provided for asomewhat asymmetric and sheared PSF. The end result will be a somewhatsharper image for a moving object at a particular distance from thelens, at the costs of somewhat less sharp image at other distances.

A standard PSF may be used, in which case it is not necessary to do anycalculation to obtain the PSF, or, in embodiments wherein the PSF isfinetuned the settings of the diffuser driver 6 may may also be an inputfor de deconvolutor 20. In FIG. 2 this is schematically indicated byblock 21 which gets some data from diffuser driver 6 to fine tune thePSF. Although the invention allows a single PSF function for a widerange of object speeds to be used, some second order effects are stillpossible wherein the PSF can be fine-tuned for a particular speed. Apossible field of application would be speed camera in such embodimentsa good estimation of the direction and the speed of an object is knownbeforehand.

In step 22 the PSF for the particular blurring action is calculated orestimated or set (if, for instance, there are a number of possiblechoices). From the point spread function (step 21) the inverse pointspread function IPSF is calculated. The blurred image 10 is deconvolutedwith the IPS in step 24 to provide a sharpened image 11.

FIG. 3 schematically illustrates the step of acquiring an image. In step31 the start button is pressed. Of course, the phrase “press button” ismerely a figure of speaking to denote the start of the image aquiringprocess. The image acquiring process may be started differently. Forinstance, for a speed camera or a surveillance camera, “press thebutton” may means that the recording process is initiated after somemovement (or movement above a certain speed limit) is detected by meanssuch as a laser gun or infrared movement sensors. In step 32 the shutteris opened and image acquisition is started. Of course, ‘shutter’ may bea physical shutter or an electronic shutter and is in FIG. 2 used todenote any process that allows light to go to the sensor, which amountsto opening the shutter, or shields the sensor from light, i.e. closingthe shutter.

During image acquisition the diffuser is swept, i.e. the diffusingeffect is changed from nearly transparent to more diffusing.

At step 34 the shutter is closed and the image acquisition is ended. Theblurred image 10 is saved in step 35. To this image an IPSF (inversePoint Spread Function) is applied in a deconvolution step. This willlead to a sharpened image 11 which is saved.

Following figures explain the concepts of the invention.

For simplicity, image formation is considered as a 2D process (time andone spatial dimension, in the figures represented by x) in thefollowing. However, the concepts discussed in this document extend totwo lateral spatial dimensions (x, y).

To better understand the concept of motion-invariant imaging, it isuseful to first outline spatio-temporal sampling characteristics ofconventional cameras. In a conventional optical imaging system, such asa photo camera, the camera is focused at an object of interest, whichamounts to fixing the displacement between the sensor and lens at anappropriate distance. Perfect static and motion sharpness can then beobtained when the spatio-temporal shutter operation is infinitesimallysmall in space and time (a Dirac δ(x,t), i.e. a singular point in spaceand time). In space time a static object remains at its position all thetime and thus at a fixed position all of the time. The exposure isinfinitesimally small in time and in space, so it is an infinitesimallysmall point in space and time. For this idealized hypothetical shutterconfiguration, the sensor records all frequencies at the same intensity,resulting in identical frequency characteristics for different motions.The Fourier transform such a signal is 1 for all values. For a Diracfunction, all frequencies are sampled equally and the amplitude at allwavenumbers is equal. Given that all frequencies are sampled equally theamplitude at all wavenumbers at all possible speeds is equal, i.e. thePSF function is a constant. The PSF functions are thus also equal. EqualPSF functions means that at all speeds of an object the image is equallysharp and can be reconstructed by deconvoluting the image. Thus an idealDirac shutter would allow a sharp image at all speeds. However, aperfect Dirac shutter cannot be constructed and realistic approximationsof it collect insufficient light to create a proper image. The image maybe sharp for all speeds of an object in the image, but not enough lightis captured to make the objects visible, which is highly impractical, tosay the least. In practice, the shutter time (and aperture) is adjustedto the local light conditions and longer exposure times are required tocollect sufficient light in dark environments to maintain an adequatesignal-to-noise ratio. One way of solving the problems would be todevelop sensors that are more sensitive to light, allowing the exposuretime to become shorter and shorter approaching a Dirac function.However, in reality limits are posed on the sensitivity of the sensorand the shutter must be kept open for a period of time enough to collectlight rays. Also the price of the camera usually increases shaprly forfaster sensors.

Most traditional cameras feature an adjustable shutter and aperture thatwindows the light coming through the lens in the temporal (i.e. duringan exposure time) and spatial dimensions. These can typically becharacterized as box filters (i.e. a constant sensitivity over a finiteinterval) in real time and space dimensions, corresponding to a sincmodulation in the corresponding temporal and spatial frequency domains.As a result, some high frequencies are fully suppressed duringacquisition and cannot be recovered during inverse FIR filtering evenwhen perfect motion information would be available. A realistic shutterhas a finite extension, thus the box has a width in the x-direction. Theshutter is open during a shutter time t.

In the temporal domain and the spatial frequency domain some highfrequencies are suppressed. Some details are therefore lost and someartifacts are created. Furthermore, since for a realistic shutter thePSF functions are different for different speeds, one cannot use asingle IPSF for deconvolution of the recorded image. In practice and intheory this means that it is impossible to deconvolute by some inversefiltering for a wide range of speeds. In short, motion invariant imagingis impossible.

For global motion, motion blur can be prevented by tracking the motionwith the camera. Alternatively, this can be achieved by moving thesensor relative to the main lens at the desired speed along a lateraltrajectory (perpendicular to the optical axis) during the exposure time.However, only objects moving at the target speed will be sharp in therecorded image, while all other scene areas remain blurred or becomeeven more blurred than if nothing would have been done.

None of the prior art documents or techniques allow an effective motionblur reduction, unless the motion or the direction of the motion wouldbe known in advance, or the camera is moved or motion vectors can beestablished, which is often not the case.

It is remarked that in International Patent Application WO 2010/131142 asystem is described in which a sweeping focus set-up is used toaccomplish motion invariant imaging. This is achieved by sweeping thefocus fast by moving the sensor or lens, of changing the focus of thelens.

Since a sweeping focus is used the acquired image is sharp throughoutthe focal sweep range, background and foreground are sharp. To the humaneye such an all-in-focus image often looks unnatural. Also the focusingcharacteristics of the camera have to be changed during acquisition.

The present invention takes a different path by introducing in the lightpath a dynamic diffuser.

FIG. 4 illustrates the function of the diffuser 6.

The diffuser 4 distributes a light ray in a disk way (i.e. beingprojected on a screen it will give a bright disk of certain radiusR_(blur)) such that the variance in the angles of the out-coming lightrays depends on the applied driving signal E from the diffuser driver 6.FIG. 5A schematically show the angle of the light disk as a function ofthe driver signal E. The larger the driver signal E is, the larger theangle α and thus the larger the blur radius R_(blur) of the light diskon the sensor. The driver signal goes from 0 to E_(max) and thus theangle goes from 0, or more precisely a very small disk, to a diskcorresponding with a maximum diffusing angle α(E_(max)). The driversignal E(t) is synchronized with the exposure, so that during theexposure time the extent of the diffuse disk goes from large to nearlyzero and then to large again.

The blur radius, which is the radius of the blurred spot on the sensoris a function of the distance s between the diffuser and the sensor,which distance may be time dependent, thus s=s(t), and of the angle α,in formula

R _(blur,diffuser)(t)=s(t)×sin(α(E(t)).

The blur radius thus changes as a function of time, if the distance sand/or the dirver signal E changes. This change is called the blur speedwhich can be calculated by

v _(blur,diffuser) =dR _(blur,diffuser) /dt=d(s(t)×sin(α(E(t)))/dt

For a constant distance s and small values of α and a linear relationbetween α and E (thus dα/dE=constant), the above formula becomesv_(blur,diffuser)=C*dE/dt, where C=s*constant and wherein C can becalculated or experimentally determined.

FIG. 6 illustrates, as a function of time t on the horizontal axis, theshape of the corresponding disk of light rays on the image plane IP,i.e. the disk that hits the sensor.

The disk goes from the start position t=t₀ (where E=E_(max) and thus alarge blur disk), to nearly a point (midway during the exposure) andthen increases again to a large disk at t=t_(end). The points of the twocones, which meet at the point of the image plane, are shown darker toillustrate that when the blur disk is small the intensity is high, sinceall light is concentrated on a small area.

The total blur kernel for the exposure is given by the summation of alllight impinging on the sensor during the exposure time. This will mainlybe concentrated near the points where the cones cross the image plane.FIG. 7 the image captured by the sensor is a superposition of thedifferently blurred images corresponding to the different values of thedistribution angle. The integral blurring effect can be modelled as aconvolution with a integral blurring kernel, illustrated on FIGS. 6, 7and 8 where FIG. 6 shows a space-time representation of the sensorintegration area, FIG. 7 illustrates the effective system blurringkernel, and FIG. 8 illustrates system blurring kernel on the logarithmicplot. If one knows the blur kernel, the blurred imaged can bedeconvoluted to provide a sharp image by a deconvoluting with the IPSFcorresponding with the integrated blur kernel. If the used IPSf is closeto the IPSF for the real situation, one will obtain a good result, i.e.a nearly sharp image. During image acquisition it is not necessary tochange the focussing properties of the camera.

The blurring kernel shown in FIGS. 6 to 8 is that for a stationaryobject. When the object moves in the image plane during the exposure theblurring kernel changes. FIGS. 9, 10 and 11 show the result of a movingobject. The cones are now situated on a sloped line. The slope of thisline is related to the object speed, i.e. the speed with which theobject moves in the image plane. This object speed on the sensor is afunction of the speed of the object in the object plane and the ratiobetween the object distance and the distance between lens and sensor.

v _(object) =dx/dt*(distance object to lens/distance lens sensor).

Where dx/dt is the object speed in the image plane, if the object ismoving in the image plane in the x-direction.

If the object is moving in the y direction dx/dt is replaced by dy/dtand if the object is moving in both directions the object speed is thesquare root of the squares of the speed in the x and the speed in they-direction, as is well known.

The further an object is away from the lens, the lower the object speedon the sensor, the closer an object is to the lens, the higher theobject speed on the sensor. The inventors have found that he system blurkernel is practically motion invariant if the object speed projected onthe sensor is below 80 percent of the blur speed v_(blur,diffuser). Inpractice this means that one can use the blur kernel for a stationaryobject as shown in FIG. 8, i.e. one that does not move, for the blurkernel for a moving object as shown in FIG. 11, without introducing inthe deconvoluted image, appreciable motion blur. Since blur kernels ofFIGS. 8 and 11 are nearly identical, the corresponding PSF functions arenearly identical, and deconvolution of recorded images using a singleIPSF function is possible and allows a sharp image to be obtained forall objects at object speeds on the sensor below the blur speed,preferably below 80% of the blur speed. In FIG. 11 this is schematicallyindicated by the dotted line along the X-axis which dotted indicates thedistance the centre of the image of the object travels over the sensorduring the exposure. The distance is comparable to the extent of theblur kernel of FIG. 8.

Apart from motion blur there may be other sources of blur, so inpractice a somewhat higher value of 100% of the blur speed is oftenacceptable.

Around a situation where the object speed projected on the sensor isequal to the blur speed one can observe a gradual transition to thesinc-like kernel which is characteristic for a traditional camera. Seee.g. FIGS. 12 and 13 which illustrate a situation in which the objectspeed on the sensor is much greater than the blur speed. The distancethe centre of the spot of the image of an object exceeds the extent ofthe blur kernel of FIG. 8.

In such situations using the blur kernel for a stationary object todeconvolute a fast moving object will not lead to good results, sincethe corresponding IPSF functions differ greatly.

Typical speeds are 5 km/hour at an object-lens distance of 2 meter of 50km/hour at an object-lens distance of 10 to 20 meter.

In preferred embodiments of the invention the diffuser driver 6 has aninput which provides information on the object speed and or the objectdistance to the lens. An example is a speeding camera, which, dependingon the street where it is used, may be triggered by a speed of 35km/hour, or a speed of 130 km/hour and may have varying distances to theposition at which the speed is to be measured. In such circumstances itmay be usefull to have different settings for the diffuser driver.Likewise a camera may have settings for different speeds.

It is remarked that within the framework of the invention “motioninvariant imaging” is not to be so strictly interpreted as to mean thatfor any speed at any level of detail there would not be a difference inimaging; the object of the invention is to reduce motion variance, i.e.motion blur, within practical limits; a perfect solution is an ideal,not the reality.

The inventors have realized that the maximum object speed for which thePSF functions of an object captured on the sensor is basically the sameas that for a static object, and thus motion invariant imaging ispossible, if the blur speed is larger than the object speed, preferablymore than 125% of the object speed on the sensor.

It should be noted that, although the above described methods andapparatuses can work blindly without having to know anything about theoccuring object speeds in the scene and consequently on the sensor, ifone has information regarding those increased reconstruction precisioncan be achieved (i.e. sharper/better final images). This can be doneeither statically (e.g. one knows which typical speeds occur, e.g. in amachine vision application where one knows the speed of the conveyerbelt upon which objects to be analyzed come by, or one guesses what thebest settings are and apply such setting), or dynamically, in which thesystem (e.g. iteratively) measures the speeds of the objects in thescene and/or the distance to the lens and adjusts the parameters of thediffuser driver optimally.

Thus, in such preferred embodiments, the system comprises means to sete.g. manually the parameters of the diffuser driver and/or means toestablish the speed and/or the distance to the camera of an object to berecorded and wherein the means for adjusting the diffuser driver arearranged to obtain a signal from the means to establish the speed and/ordistance of an object to be recorder.

In embodiments the diffuser is a dynamical diffuser to which a driversignal is sent to change the diffusing properties of the diffuser.

FIGS. 14A and 14B show embodiments which have the required properties.In both of them the diffuser comprises a layer of Liquid Crystals (LC)125 enclosed between two transparent electrodes, e.g. made oftransparent ITO, 123 and 124. This arrrangement is positionbed betweentwo transparent glass or plastic substrates 121, 122. The LC arebi-refringent. Hence the local misalignment of LC molecules createsdiffraction effect. At the absence of electric field the LC are alignedthe same way, according to the surface of the electrodes. This makes thediffusers completely transparent. Between the electrodes a voltage maybe applied. To that end the electrodes are connected to means forapplying a voltage between the electrodes. When the voltage is applied,the electrodes 123, 124 create an electric field with therandom/irregular local alignments. The LC rotates in the direction ofthe local electric field. Since LC are randomly/irregular aligned, thelocal refraction effects are perceived as a diffusive effect atmacro-scale. Two embodiments shown on FIGS. 14A and 14B differ in theimplementation of the non-uniform electric field. In FIG. 14A we proposeto use two planar ITO electrodes and a planar electric field modulatinglayer 126 which has uniform refraction index and non-uniform dielectrictransparency index. This layer can be made by combining two materials(e.g. encapsulating one into another) with identical refraction indicesand different di-electric properties. If the voltage is applied to theplanar electrodes the modulating layer is responsible for the creationof non-uniform electric field. The second embodiment is illustrated inFIG. 14B. In this embodiment at least one planar electrode 122, 124 isreplaced by a grid electrode 127 which can e.g. be printed on the glassor plastic substrate. The grid size should be comparable with thethickness of LC material in order to create sufficient non-uniformity ofthe electric field. The grid should be sufficiently regular in order toprovide uniform blurring effect on the macro-scale. Two examples of theplanar grid electrodes are illustrated in the FIGS. 15A and 15B.

It will be clear that the exemplatry embodiments of the invention aregiven by means of example and do not restrict the invention to theexamples given.

For instance: Usually an image is taken in visible light. However,within the frameworkf of the invention the image may also be taken ininfrared.

The deconvolutor 20 may form a part of the camera, or the deconvolutoris a part of a personal computer or is situated on a site on theinternet to which one sends the images 10 for deconvolution. In thelatter embodiment one could send image data comprising, for instance asmeta-data, parameters to be used in the deconvolution step 23, i.e. thesettings of the means 6. An example of such a set-up would be a grid ofspeed cameras which can be set for various speed limits, depending onthe local speed limit and/or local circumstances, such as the amount oftraffic, weather conditions, for instance fog, or activities such asroad repair. Each speed camera may have its own setting depending on thetype or make of camera, and which which may be even adjustable tocircumstances, i.e. the then and there applicable speed limit. Thecameras send their image data to a central department, and sen with theimage data the there and then applicable speed limit, and or thesettings of means 6. At a regional or national or even transnationalprocessing location (which may be an internet site) the image data withmeta data are received, the sharp images are made, and these are addedto the speeding tickets.

The invention may be used to take fotopgraph or for video, for 2Dimages, or for taking 3D images. There can be used a single diffuser ortwo diffuser in series. The position of the diffuser may be in betweenthe lens and the sensor, or anywhere else in the light path. Thediffuser may be integrated in a lens system.

Since the kernel is in effect due to a sum of more or less diffusedimages, on can also use time multiplexing wherein the diffuser is duringthe exposure time multiplexed between a transparent and diffusing state,wherein the ratio between the two states is a function of time, insynchronicity with the exposure, ranging from highly diffuse at t=t₀, tocompletely transparant at the middel of the exposure, to highly diffuseat t=t_(end). The dynamic behaviour of the diffuser may be initiatedslightly before the shutter is opened and continue to slightly afterclosing of the shutter. This will remove any start-up irregularities.

In the examples the blur speed is taken to be constant. In embodimentsthe blur speed may be non-linear to emphasize more the middle part ofthe exposure, or either end of the exposure.

As explained, in embodiments it is possible to set the parameters of theblurring action. In embodiments the camera could allow various manualsettings dependent on how fast it is assumed that the object moves, orhow fast it is measured to be, wherein during exposure the position ofthe diffuser is static, but prior to the exposure the diffuser is movedto a certain position. As explained above the diffusing action isdependent on the distance s between the diffuser and the sensor, so,with one and the same relationship between the angle a and the signal E,one can expand the maximum extent of the blur radius during exposure (toexten the range of motion invariance, by moving the diffuser closer toor further away from the sensor. During the exposure, however, thediffuser does not move, so there is no movement during exposure, theonly thing that is required is that the diffuser is moved prior toexposure.

In short the invention can be described by:

A system and camera wherein the camera comprises in the light path adiffuser (4). The system or camera comprises a means (6) to modulate thediffusing properties of the diffuser (4) on an image projected by thelens on the sensor during exposure of the image. To the captured blurredimage (10) an inverse point spread function is applied to deconvolute(24) the blurred image to a sharper image. Motion invariant image can sobe achieved.

The deconvolution can be performed inside the camera or camera system.

1. A camera system comprising: a camera with a lens and an image sensor;a liquid crystal diffuser in a light path of the camera; a diffuserdriver that electronically drives the liquid crystal diffuser to vary anamount of diffusion by the liquid crystal diffuser on light received bythe camera system projected by the lens on the sensor during anindividual exposure of an image on the sensor; and a controller thatsynchronizes the diffuser driver and the amount of diffusion with theexposure to vary the amount of diffusion during the exposure.
 2. Acamera system as claimed in claim 1, wherein the diffuser driverdynamically changes diffusing properties of the liquid crystal diffuserin synchronicity with the exposure.
 3. A camera system as claimed inclaim 1, wherein the diffuser driver is arranged to, during theindividual exposure, drive the liquid crystal diffuser gradually from adiffusing state, to a lesser diffusing state, to a transparent state, toa lesser diffusing state, to a diffusing state.
 4. A camera system asclaimed in claim 3, further comprising a controller configured to causethe diffuser driver to electronically drive the liquid crystal diffuser.5. A camera system as claimed in claim 4 wherein the controller isconfigured such that parameters of the diffuser driver are set manually.6. A camera system as claimed in claim 4, wherein the diffuser driver isconfigured to establish the speed and/or the distance to the camera ofan object to be recorded and wherein the controller is arranged toobtain a signal from the diffuser driver that communicates informationrelated to the speed and/or the distance to the camera of the object. 7.A camera system as claimed in claim 6, further comprising a deconvolutorfor deconvolution of a recorded image, wherein the recorded image isdeconvoluted with an inverse point spread function (IPSF).
 8. A camerasystem as claimed in claim 7 wherein the liquid crystal diffuser ispositioned in front of the lens.
 9. A camera system as claimed in claim7 wherein the liquid crystal diffuser is positioned in between the lensand the sensor.
 10. A camera for a camera system as claimed in claim 9.11. A method of operating a camera, wherein the camera comprises a lens,an image sensor, and a liquid crystal diffuser in a light path of thecamera, the method comprising: electronically driving the liquid crystaldiffuser during an individual exposure for image acquisition to vary anamount of diffusion by the liquid crystal diffuser on light received bythe camera system projected by the lens on the sensor; and synchronizingthe amount of diffusion with the exposure to vary the amount ofdiffusion during the exposure.
 12. A method as claimed in claim 11,wherein the amount of diffusion depends on signals conveying informationcorresponding to a speed and/or a distance to the lens of a recordedobject.
 13. A method for deconvoluting image data recorded by a cameracomprising a lens and an image sensor and comprising in a light path ofthe camera a liquid crystal diffuser, wherein during image acquisitionthe liquid crystal diffuser is electronically driven to vary an amountof diffusion by the liquid crystal diffuser on light received by thecamera system projected by the lens on the sensor during an individualexposure of an image on the image sensor, and an inverse point spreadfunction (IPSF) is applied to the acquired image to deconvolute theacquired image.
 14. A camera system as claimed in claim 1 wherein thesystem comprises a means for moving the diffuser in synchronicity withthe exposure.